A Comparison of Non-informative Priors for Bayesian Networks Produced as Part of the Esprit Working Group in Neural and Computational Learning Ii, Neurocolt2 27150

We consider Bayesian and information-theoretic approaches for determining non-informative prior distributions in a parametric model family. The information-theoretic approaches are based on the recently modiied deenition of stochastic complexity by Rissanen, and on the Minimum Message Length (MML) approach by Wallace. The Bayesian alternatives include the uniform prior, and various equivalent sample size priors. In order to be able to empirically compare the diierent approaches in practice, the methods are instantiated for a model family of practical importance, the family of Bayesian networks. The results with several public domain datasets show that the choice of the prior distribution can have a significant eeect on the results obtained, especially if the amount of the data available is small. Inspired by our empirical observations, we also introduce a new heuristics for determining the prior distribution. The empirical results show that the heuristics gives consistently very good results with respect to the results obtained by alternative methods.